Problem solving team interventions: Web-based and consultant-based experimental evaluations.

This research was designed to develop, implement, and evaluate an assessment and intervention protocol to increase problem-solving teams' (PSTs) adoption and implementation of evidence-based practices aimed at students with disruptive behavior problems. Participants included 15 PSTs. Adopting single-case design methodology, we examined whether a customized set of assessment and intervention consultant-led intervention procedures could be used to improve the activities, process, and recommendations of PSTs compared to a web-based intervention. We were interested in evaluating two variations of the problem-solving model based on the team initiated problem-solving (TIPS) approach. TIPS includes steps to successful problem solving and solution implementation for student academic and behavioral concerns. Based on visual analysis and statistical randomization tests, we found that a teleconsultation web-based model of PST intervention was not effective in improving the functioning of the PST. In contrast, a customized, consultation-led intervention model with PST facilitators that followed this approach was found to be effective in improving both the foundation and thoroughness of the PST's problem solving. Implications of future PST improvement models for practice and research are discussed.

Teaching argument writing in math class: challenges and solutions to improve the performance of 4th and 5th graders with disabilities.

Incorporating argument writing as a learning activity has been found to increase students' mathematics performance. However, teachers report receiving little to no preservice or inservice preparation to use writing to support students' learning. This is especially concerning for special education teachers who provide highly specialized mathematics instruction (i.e., Tier 3) to students with mathematics disabilities (MLD). The purpose of this study was to examine the effectiveness of teachers providing content-focused open-ended questioning strategies, which included both argument writing and foundational fraction content, using Practice-Based Professional Development (PBPD) and Self-Regulated Strategy Development (SRSD) for implementing a writing-to-learn strategy called FACT-R2C2. We report the relative number of higher-order mathematical content questions that teachers asked during instruction, from among three different-level question types: Level 1: yes/no questions focused on the mathematics content; Level 2: one-word responses focused on the mathematics content; and Level 3: higher-order open-ended responses centered around four mathematical practices from the Common Core State Standards for Mathematics. Within a well-controlled single-case multiple-baseline design, seven special education teachers were randomly assigned to each PBPD + FACT-R2C2 intervention tier. Results indicated that: (1) teachers' relative use of Level 3 questions increased following the introduction of the FACT intervention; (2) this increase was apart from the professional development training that the teachers had initially received; and (3) students' writing quality improved to some extent with the increase in teachers' relative use of Level 3 questions. Implications and future directions are discussed.

Does Conceptual Transparency in Manipulatives Afford Place-Value Understanding in Children at Risk for Mathematics Learning Disabilities?

We investigated the effect of conceptual transparency in the physical structure of manipulatives on place-value understanding in typically developing children and those at risk for mathematics learning disabilities. Second graders were randomly assigned to one of three manipulatives conditions: (a) attachable beads that did not make the denominations or ones in the denominations transparent, (b) pipe cleaners that made only the denominations transparent, and (c) string beads that made both the denominations and the ones in the denominations transparent. Participants used the manipulatives to represent double- and triple-digit numerals. Statistical analyses indicated that the transparency of the denominations, but not the transparency of the ones in the denominations, is responsible for children's number representation and place-value understanding. Descriptive analyses of their responses revealed that the at-risk children were at a greater disadvantage than their typically developing peers with the attachable beads, failing to use place-value concepts to interpret their representations.

Single-case intervention research design standards: Additional proposed upgrades and future directions.

Single-case intervention research design standards have evolved considerably over the past decade. These standards serve the dual role of assisting in single-case design (SCD) intervention research methodology and as guidelines for literature syntheses within a particular research domain. In a recent article (Kratochwill et al., 2021), we argued for a need to clarify key features of these standards. In this article we offer additional recommendations for SCD research and synthesis standards that have been either underdeveloped or missing in the conduct of research and in literature syntheses. Our recommendations are organized into three categories: expanding design standards, expanding evidence standards, and expanding the applications and consistency of SCDs. The recommendations we advance are for consideration for future standards, research design training, and they are especially important to guide the reporting of SCD intervention investigations as they enter the literature-synthesis phase of evidence-based practice initiatives.

Randomization Procedures for Changing Criterion Designs.

To strengthen the scientific credibility arguments for single-case intervention studies, randomization design-and-analysis methods have been developed for the multiple-baseline, ABAB, and alternating treatment designs, including options for preplanned designs, wherein the series and phase lengths are established prior to gathering data, as well as options for response-guided designs, wherein ongoing visual analyses guide decisions about when to intervene. Our purpose here is to develop randomization methods for another class of single-case design, the changing criterion design. We first illustrate randomization design-and-analysis methods for preplanned changing criterion designs and then develop and illustrate methods for response-guided changing criterion designs. We discuss the limitations associated with the randomization methods and the validity of the corresponding intervention-effect inferences.

Examining and Enhancing the Methodological Quality of Nonconcurrent Multiple-Baseline Designs.

In this article we respond to the recent recommendation of Slocum et al. (2022), who provided conceptual and methodological recommendations for reconsidering the credibility and validity of the nonconcurrent multiple-baseline design. We build on these recommendations and offer replication and randomization upgrades that should further improve the status of the nonconcurrent version of the design in standards and single-case design research. Although we suggest that the nonconcurrent version should be an acceptable methodological option for single-case design researchers, the traditional concurrent multiple-baseline design should generally be the design of choice.

Academic and Behavior Combined Support: Evaluation of an integrated supplemental intervention for early elementary students.

In current multi-tier models, students who are not responsive to universal instruction receive supplementary support. Despite most students having co-occurring academic and behavior challenges, their needs are often addressed through separate academic or behavior interventions. This approach may not only be costly for schools, but it also fails to acknowledge the well-documented link between behavior and academic performance. The purpose of this study was to evaluate Academic and Behavior Combined Support (ABC Support), a newly developed supplemental intervention that merges a combined focus on reading fluency and engagement. Six teachers implemented the intervention for 6-8 weeks with Grade 2 students. Students' oral fluency on standard and training reading passages and occurrence of engagement and disruptive behaviors during classroom reading instruction were measured repeatedly across baseline and intervention phases in a multiple-baseline design. Visual and statistical analyses revealed significant improvement from baseline to intervention for both reading and behavior outcomes. Post-intervention surveys and interviews revealed a high level of acceptability among teachers and students. Theoretical and empirical contributions as well as practice implications are addressed.

Single-case design standards: An update and proposed upgrades.

In this paper, we provide a critique focused on the What Works Clearinghouse (WWC) Standards for Single-Case Research Design (Standards 4.1). Specifically, we (a) recommend the use of visual-analysis to verify a single-case intervention study's design standards and to examine the study's operational issues, (b) identify limitations of the design-comparable effect-size measure and discuss related statistical matters, (c) review the applicability and practicality of Standards 4.1 to single-case designs (SCDs), and (d) recommend inclusion of content pertaining to diversity, equity, and inclusion in future standards. Within the historical context of the WWC Pilot Standards for Single-Case Design (1.0), we suggest that Standards 4.1 may best serve as standards for meta-analyses of SCDs but will need to make clear distinctions among the various types of SCD studies that are included in any research synthesis. In this regard, we argue for transparency in SCD studies that meet design standards and those that do not meet design standards in any meta-analysis emanating from the WWC. The intent of these recommendations is to advance the science of SCD research both in research synthesis and in promoting evidence-based practices.

Different randomized multiple-baseline models for different situations: A practical guide for single-case intervention researchers.

Single-case researchers often implement multiple-baseline designs as their preferred methodology for intervention evaluations. Recent writings and empirical investigations have argued in favor of incorporating various forms of randomization into such designs for the purpose of elevating the intervention study's internal validity and scientific credibility. In this article, we consider a variety of randomized multiple-baseline designs and associated randomization statistical tests, along with their potential strengths and limitations. In what amounts to a practical guide, we refer school psychology researchers to these versatile randomization procedures for planning and executing their intervention studies.

Randomized Single-Case Intervention Designs and Analyses for Health Sciences Researchers: A Versatile Clinical Trials Companion.

AIM: An expository note introduces health sciences researchers to randomized single-case intervention designs, an adaptation of interrupted time-series methodology, and the staple of a scientifically credible small-sample research paradigm. METHODS: Detailed examples illustrating two different randomized single-case procedures are presented to highlight the techniques' advantages relative to small-sample nonparametric procedures that are commonly applied in the medical and health sciences fields. RESULTS: Numerous positive outcomes, based on both statistical simulation studies and actual intervention research investigations, support the applicability and value of these procedures. CONCLUSION: Randomized single-case intervention designs are recommended for consideration by health sciences researchers.

Randomization procedures in single-case intervention research contexts: (Some of) "the rest of the story".

Following up on articles recently published in this journal, the present contribution tells (some of) "the rest of the story" about the value of randomization in single-case intervention research investigations. Invoking principles of internal, statistical-conclusion, and external validity, we begin by emphasizing the critical distinction between design randomization and analysis randomization, along with the necessary correspondence between the two. Four different types of single-case design-and-analysis randomization are then discussed. The persistent negative influence of serially dependent single-case outcome observations is highlighted, accompanied by examples of inappropriate applications of parametric and nonparametric tests that have appeared in the literature. We conclude by presenting valid applications of single-case randomization procedures in various single-case intervention contexts, with specific reference to a freely available Excel-based software package that can be accessed to incorporate the present randomization schemes into a wide variety of single-case intervention designs and analyses.

Evaluating implementation supports delivered within problem-solving consultation.

Teachers participating in problem-solving consultation often struggle to maintain adequate treatment fidelity, which is necessary to improve student outcomes. Low levels of treatment fidelity may result from implementation barriers, such as intervention compatibility, implementer skill, and implementer motivation. This study involves the evaluation of five implementation supports designed to address implementation barriers (i.e., Implementation Planning, Role Play, Participant Modeling, Raising Awareness, Motivational Consulting) within problem-solving consultation. Across 14 randomized individual single-case AB intervention designs, we evaluated the impact of these implementation supports on teacher treatment fidelity of classroom management plans and class-wide academic engagement and disruptive behavior. Visual analysis, descriptive statistics, and randomization test analyses suggest that these implementation supports have the potential to be broadly effective in improving teachers' fidelity and student outcomes. Teachers required a different number of supports to increase fidelity levels and rated the implementation supports positively. Implications of the study's findings are described.

A Note on the "One Case per Stagger Position" Character of Common Multiple-Baseline Intervention Designs.

PURPOSE: To overcome the constraint of common multiple-baseline designs that only one case per stagger position is permitted. METHODS: Three alternative strategies for assigning more than one case to each stagger position are examined. RESULTS: The three recommended strategies achieve the objective while maintaining the study's internal and statistical-conclusion validities. CONCLUSIONS: ExPRT, a freely available Excel-based randomization-test package, can be used to assist in both the design and statistical analysis associated with each of the strategies.

Comparison of randomization-test procedures for single-case multiple-baseline designs.

In three simulation investigations, we examined the statistical properties of several different randomization-test procedures for analyzing the data from single-case multiple-baseline intervention studies. Two procedures (Wampold-Worsham and Revusky) are associated with single fixed intervention start points and three are associated with randomly determined intervention start points. Of the latter three, one (Koehler-Levin) is an existing procedure that has been previously examined and the other two (modified Revusky and restricted Marascuilo-Busk) are modifications and extensions of existing procedures. All five procedures were found to maintain their Type I error probabilities at acceptable levels. In most of the conditions investigated here, two of the random start-point procedures (Koehler-Levin and restricted Marascuilo-Busk) were more powerful than the others with respect to detecting immediate abrupt intervention effects. For designs in which it is not possible to include the same series lengths for all cases, either the modified Revusky or restricted Marascuilo-Busk procedure is recommended.

Pictorial mnemonic-strategy interventions for children with special needs: Illustration of a multiply randomized single-case crossover design.

An innovative single-case crossover design containing multiple forms of randomization was implemented with eight participants in seven weekly sessions, during which instruction was given in the use of two different pictorial mnemonic (memory-enhancing) strategies: one designed to improve the children's learning of the dates of various inventions and the other designed to improve the children's acquisition of unfamiliar vocabulary items. A composite randomization statistical test revealed that when compared with the children's own preferred learning methods, the mnemonic-strategy approach produced the predicted facilitation effects. At the same time, it was evident that mnemonic instruction enhanced children's performance to a greater extent on the vocabulary task than on the inventions task. In-depth examination of both individual student performance profiles and the tasks/procedures were conducted, yielding recommendations and challenges for follow-up single-case intervention research on the topic.

A Monte Carlo evaluation of masked visual analysis in response-guided versus fixed-criteria multiple-baseline designs.

We developed masked visual analysis (MVA) as a structured complement to traditional visual analysis. The purpose of the present investigation was to compare the effects of computer-simulated MVA of a four-case multiple-baseline (MB) design in which the phase lengths are determined by an ongoing visual analysis (i.e., response-guided) versus those in which the phase lengths are established a priori (i.e., fixed criteria). We observed an acceptably low probability (less than .05) of false detection of treatment effects. The probability of correctly detecting a true effect frequently exceeded .80 and was higher when: (a) the masked visual analyst extended phases based on an ongoing visual analysis, (b) the effects were larger, (c) the effects were more immediate and abrupt, and (d) the effects of random and extraneous error factors were simpler. Our findings indicate that MVA is a valuable combined methodological and data-analysis tool for single-case intervention researchers.

Additional comparisons of randomization-test procedures for single-case multiple-baseline designs: Alternative effect types.

A number of randomization statistical procedures have been developed to analyze the results from single-case multiple-baseline intervention investigations. In a previous simulation study, comparisons of the various procedures revealed distinct differences among them in their ability to detect immediate abrupt intervention effects of moderate size, with some procedures (typically those with randomized intervention start points) exhibiting power that was both respectable and superior to other procedures (typically those with single fixed intervention start points). In Investigation 1 of the present follow-up simulation study, we found that when the same randomization-test procedures were applied to either delayed abrupt or immediate gradual intervention effects: (1) the powers of all of the procedures were severely diminished; and (2) in contrast to the previous study's results, the single fixed intervention start-point procedures generally outperformed those with randomized intervention start points. In Investigation 2 we additionally demonstrated that if researchers are able to successfully anticipate the specific alternative effect types, it is possible for them to formulate adjusted versions of the original randomization-test procedures that can recapture substantial proportions of the lost powers.

Meta- and statistical analysis of single-case intervention research data: quantitative gifts and a wish list.

In this commentary, we add to the spirit of the articles appearing in the special series devoted to meta- and statistical analysis of single-case intervention-design data. Following a brief discussion of historical factors leading to our initial involvement in statistical analysis of such data, we discuss: (a) the value added by including statistical-analysis recommendations in the What Works Clearinghouse Standards for single-case intervention designs; (b) the importance of visual analysis in single-case intervention research, along with the distinctive role that could be played by single-case effect-size measures; and (c) the elevated internal validity and statistical-conclusion validity afforded by the incorporation of various forms of randomization into basic single-case design structures. For the future, we envision more widespread application of quantitative analyses, as critical adjuncts to visual analysis, in both primary single-case intervention research studies and literature reviews in the behavioral, educational, and health sciences.

Nonparametric statistical tests for single-case systematic and randomized ABAB AB and alternating treatment intervention designs: new developments, new directions.

In this four-investigation Monte Carlo simulation study, we examined the properties of nonparametric randomization and permutation statistical tests applied to single-case ABAB…AB and alternating treatment designs based on either systematically alternating or randomly determined phase assignments. Contrary to previous admonitions, when appropriately conceptualized nonparametric tests were conducted the systematic designs maintained Type I error rates at or less than their nominal values whenever either no or positive autocorrelation was present in the series. Statistical power for all designs varied as a function of both the number of phases and the number of observations per phase. It was concluded that to increase methodological rigor and statistical precision in the face of autocorrelation, (a) whenever possible, researchers should adopt a randomization scheme in assigning the orders in which the A and B phases are administered and (b) in situations where randomized phase assignment is not feasible, researchers should adopt a systematic phase assignment scheme consisting of individually alternating A and B observations, with a cost of sacrificing some degree of statistical power. Implications for single-case intervention research in school psychology are discussed.

Extensions of a versatile randomization test for assessing single-case intervention effects.

The purpose of the present study was to investigate the statistical properties of two extensions of the Levin-Wampold (1999) single-case simultaneous start-point model's comparative effectiveness randomization test. The two extensions were (a) adapting the test to situations where there are more than two different intervention conditions and (b) examining the test's performance in classroom-based intervention situations, where the number of time periods (and associated outcome observations) is much smaller than in the contexts for which the test was originally developed. Various Monte Carlo sampling situations were investigated, including from one to five participant blocks per condition and differing numbers of time periods, potential intervention start points, degrees of within-phase autocorrelation, and effect sizes. For all situations, it was found that the Type I error probability of the randomization test was maintained at an acceptable level. With a few notable exceptions, respectable power was observed only in situations where the numbers of observations and potential intervention start points were relatively large, effect sizes were large, and the degree of within-phase autocorrelation was relatively low. It was concluded that the comparative effectiveness randomization test, with its desirable internal validity and statistical-conclusion validity features, is a versatile analytic tool that can be incorporated into a variety of single-case school psychology intervention research situations.